AutoML Leaderboard

Best model	name	model_type	metric_type	metric_value	train_time
	1_Baseline	Baseline	logloss	0.651834	1.61
	2_DecisionTree	Decision Tree	logloss	0.745252	14.68
	3_Linear	Linear	logloss	0.729557	5.14
	4_Default_Xgboost	Xgboost	logloss	0.672562	5.75
	5_Default_NeuralNetwork	Neural Network	logloss	1.1347	2.55
	6_Default_RandomForest	Random Forest	logloss	0.642638	7.02
the best	Ensemble	Ensemble	logloss	0.637739	0.65

AutoML Performance

AutoML Performance Boxplot

Features Importance

features importance across models

Spearman Correlation of Models

models spearman correlation

Summary of Ensemble

<< Go back

Ensemble structure

Model	Weight
1_Baseline	2
6_Default_RandomForest	3

Metric details

	score	threshold
logloss	0.637739	nan
auc	0.590278	nan
f1	0.590164	0.284983
accuracy	0.678571	0.475037
precision	0.666667	0.475037
recall	1	0.198183
mcc	0.282523	0.284983

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.637739	nan
auc	0.590278	nan
f1	0.307692	0.475037
accuracy	0.678571	0.475037
precision	0.666667	0.475037
recall	0.2	0.475037
mcc	0.223772	0.475037

Confusion matrix (at threshold=0.475037)

	Predicted as 0	Predicted as 1
Labeled as 0	34	2
Labeled as 1	16	4

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 5_Default_NeuralNetwork

<< Go back

Neural Network

n_jobs: -1
dense_1_size: 32
dense_2_size: 16
learning_rate: 0.05
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

1.3 seconds

Metric details

	score	threshold
logloss	1.1347	nan
auc	0.543056	nan
f1	0.533333	0.404951
accuracy	0.642857	0.57251
precision	0.5	0.57251
recall	1	4.96829e-06
mcc	0.230256	0.404951

Metric details with threshold from accuracy metric

	score	threshold
logloss	1.1347	nan
auc	0.543056	nan
f1	0.5	0.57251
accuracy	0.642857	0.57251
precision	0.5	0.57251
recall	0.5	0.57251
mcc	0.222222	0.57251

Confusion matrix (at threshold=0.57251)

	Predicted as 0	Predicted as 1
Labeled as 0	26	10
Labeled as 1	10	10

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 1_Baseline

<< Go back

Baseline Classifier (Baseline)

n_jobs: -1
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

0.5 seconds

Metric details

	score	threshold
logloss	0.651834	nan
auc	0.5	nan
f1	0.526316	0.316071
accuracy	0.357143	0.316071
precision	0.357143	0.316071
recall	1	0.316071
mcc	0	0.316071

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.651834	nan
auc	0.5	nan
f1	0.526316	0.316071
accuracy	0.357143	0.316071
precision	0.357143	0.316071
recall	1	0.316071
mcc	0	0.316071

Confusion matrix (at threshold=0.316071)

	Predicted as 0	Predicted as 1
Labeled as 0	0	36
Labeled as 1	0	20

Learning curves

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

<< Go back

Summary of 3_Linear

<< Go back

Logistic Regression (Linear)

n_jobs: -1
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

3.7 seconds

Metric details

	score	threshold
logloss	0.729557	nan
auc	0.527778	nan
f1	0.528302	0.326015
accuracy	0.642857	0.58524
precision	0.5	0.58524
recall	1	0.0337747
mcc	0.167739	0.326015

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.729557	nan
auc	0.527778	nan
f1	0.285714	0.58524
accuracy	0.642857	0.58524
precision	0.5	0.58524
recall	0.2	0.58524
mcc	0.121716	0.58524

Confusion matrix (at threshold=0.58524)

	Predicted as 0	Predicted as 1
Labeled as 0	32	4
Labeled as 1	16	4

Learning curves

Coefficients

feature	Learner_1
platelets	0.325053
serum_sodium	0.202795
diabetes	0.0449569
age	0.0179511
smoking	-0.0346126
sex	-0.0860754
anaemia	-0.08686
serum_creatinine	-0.0868632
creatinine_phosphokinase	-0.118859
ejection_fraction	-0.208866
intercept	-0.47464
DEATH_EVENT	-0.562324
time	-0.790856

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

SHAP worst decisions class 0 from Fold 1

Top-10 Best decisions for class 0 (Fold 1)

SHAP best decisions class 0 from Fold 1

Top-10 Worst decisions for class 1 (Fold 1)

SHAP worst decisions class 1 from Fold 1

Top-10 Best decisions for class 1 (Fold 1)

SHAP best decisions class 1 from Fold 1

<< Go back

Summary of 6_Default_RandomForest

<< Go back

Random Forest

n_jobs: -1
criterion: gini
max_features: 0.9
min_samples_split: 30
max_depth: 4
eval_metric_name: logloss
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

5.5 seconds

Metric details

	score	threshold
logloss	0.642638	nan
auc	0.590278	nan
f1	0.590164	0.240845
accuracy	0.678571	0.557601
precision	0.666667	0.557601
recall	1	0.11959
mcc	0.282523	0.240845

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.642638	nan
auc	0.590278	nan
f1	0.307692	0.557601
accuracy	0.678571	0.557601
precision	0.666667	0.557601
recall	0.2	0.557601
mcc	0.223772	0.557601

Confusion matrix (at threshold=0.557601)

	Predicted as 0	Predicted as 1
Labeled as 0	34	2
Labeled as 1	16	4

Learning curves

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

SHAP worst decisions class 0 from Fold 1

Top-10 Best decisions for class 0 (Fold 1)

SHAP best decisions class 0 from Fold 1

Top-10 Worst decisions for class 1 (Fold 1)

SHAP worst decisions class 1 from Fold 1

Top-10 Best decisions for class 1 (Fold 1)

SHAP best decisions class 1 from Fold 1

<< Go back

Summary of 2_DecisionTree

<< Go back

Decision Tree

n_jobs: -1
criterion: gini
max_depth: 3
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

12.9 seconds

Metric details

	score	threshold
logloss	0.745252	nan
auc	0.589583	nan
f1	0.580645	0.264344
accuracy	0.625	0.643939
precision	0.466667	0.454545
recall	1	0.0321429
mcc	0.258199	0.264344

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.745252	nan
auc	0.589583	nan
f1	0.222222	0.643939
accuracy	0.625	0.643939
precision	0.428571	0.643939
recall	0.15	0.643939
mcc	0.0563436	0.643939

Confusion matrix (at threshold=0.643939)

	Predicted as 0	Predicted as 1
Labeled as 0	32	4
Labeled as 1	17	3

Learning curves

Decision Tree

Tree #1

Rules

if (time <= 198.5) and (platelets <= 267500.0) and (creatinine_phosphokinase > 118.0) then class: 0 (proba: 72.13%) | based on 61 samples

if (time <= 198.5) and (platelets > 267500.0) and (time > 60.5) then class: 0 (proba: 54.55%) | based on 33 samples

if (time > 198.5) and (platelets <= 399500.0) and (age > 50.5) then class: 0 (proba: 96.43%) | based on 28 samples

if (time <= 198.5) and (platelets <= 267500.0) and (creatinine_phosphokinase <= 118.0) then class: 1 (proba: 55.0%) | based on 20 samples

if (time <= 198.5) and (platelets > 267500.0) and (time <= 60.5) then class: 1 (proba: 83.33%) | based on 12 samples

if (time > 198.5) and (platelets <= 399500.0) and (age <= 50.5) then class: 0 (proba: 75.0%) | based on 8 samples

if (time > 198.5) and (platelets > 399500.0) and (platelets > 436500.0) then class: 0 (proba: 100.0%) | based on 3 samples

if (time > 198.5) and (platelets > 399500.0) and (platelets <= 436500.0) then class: 1 (proba: 100.0%) | based on 3 samples

Permutation-based Importance

Confusion Matrix

Normalized Confusion Matrix

ROC Curve

Kolmogorov-Smirnov Statistic

Precision-Recall Curve

Calibration Curve

Cumulative Gains Curve

Lift Curve

SHAP Importance

SHAP Dependence plots

Dependence (Fold 1)

SHAP Dependence from Fold 1

SHAP Decision plots

Top-10 Worst decisions for class 0 (Fold 1)

SHAP worst decisions class 0 from Fold 1

Top-10 Best decisions for class 0 (Fold 1)

SHAP best decisions class 0 from Fold 1

Top-10 Worst decisions for class 1 (Fold 1)

SHAP worst decisions class 1 from Fold 1

Top-10 Best decisions for class 1 (Fold 1)

SHAP best decisions class 1 from Fold 1

<< Go back

Summary of 4_Default_Xgboost

<< Go back

Extreme Gradient Boosting (Xgboost)

n_jobs: -1
objective: binary:logistic
eta: 0.075
max_depth: 6
min_child_weight: 1
subsample: 1.0
colsample_bytree: 1.0
eval_metric: logloss
explain_level: 2

Validation

validation_type: split
train_ratio: 0.75
shuffle: True
stratify: True

Optimized metric

logloss

Training time

4.3 seconds

Metric details

	score	threshold
logloss	0.672562	nan
auc	0.554167	nan
f1	0.545455	0.486899
accuracy	0.642857	0.486899
precision	0.5	0.486899
recall	1	0.257843
mcc	0.258199	0.486899

Metric details with threshold from accuracy metric

	score	threshold
logloss	0.672562	nan
auc	0.554167	nan
f1	0.545455	0.486899
accuracy	0.642857	0.486899
precision	0.5	0.486899
recall	0.6	0.486899
mcc	0.258199	0.486899

Confusion matrix (at threshold=0.486899)

	Predicted as 0	Predicted as 1
Labeled as 0	24	12
Labeled as 1	8	12